Large-Scale Cortical Interactions During Auditory, Visual and Audiovisual Speech Processing: Event Related Analysis of Human ECoG

Based on the empirical findings on the human cortical locations responsive to auditory and/or visual stimuli, the objective of this work is to study the neural interactions within and across these cortical locations using electrocorticography (ECoG) signals. To this end, we obtain the brain functional connectivity networks under three basic speech conditions, namely auditory-only, visual-only and audiovisual. We show that the patterns of neural interactions largely depend on the condition of speech. In particular, under the auditory-only speech condition, the superior temporal gyrus directly communicates with the prefrontal cortex, and the prefrontal cortex interacts with the somatosensory cortex to process the auditory input. In the visual-only speech condition, the middle temporal gyrus and motor cortex directly exchange information, while the occipital lobe interacts with the prefrontal cortex. In the audio-visual speech condition, the prefrontal cortex communicates with the somatosensory cortex, and the superior temporal gyrus communicates with the occipital lobe to process the information. In addition, local communications within the motor cortex, somatosensory cortex, Wernick’s area and superior temporal gyrus increase under all speech conditions. Several properties of the networks, including degree and strength of each node, are used to identify the areas that play key roles in speech processing for each speech condition. These analyses reveal that the superior temporal gyrus is highly involved in the auditory-only speech processing. In the visual-only speech condition, the occipital lobe, motor cortex, somatosensory cortex and prefrontal cortex play an active role in speech processing. The properties of the brain functional network under the audio-visual speech condition confirm the supra modality of speech by integrating the properties of the brain networks in the auditory-only and visual-only speech conditions. 1. Significance Statement There exist empirical findings on the human cortical locations responsive to auditory and/or visual stimuli. It Email addresses: [email protected] (Somayeh Sojoudi), [email protected] (Werner Doyle), [email protected] (Daniel Friedman), [email protected] (Patricia Dugan), [email protected] (Orrin Devinsky), [email protected] (Thomas Thesen) is also known that the speech perception is a multi-modal process that integrates both visual and acoustic information. However, the cortical interactions across the responsive brain areas are not well understood. Using high temporal and spatial resolution data collected from ECoG electrodes, we study the large-scale cortical interactions across responsive brain regions under three basic speech conditions. This study shows the importance of shortand long-range cortical interactions and identifies the areas that play an important role in speech processing under each speech condition. In this work, the brain functional networks are obtained based on partial correlation, which shows that the brain network is sparse and connected with large communicating distances.